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Preface 


information  reported  here  on  Total  Quality  Management/ 
Methodology  for  Generating  Efficiency  and  Effectiveness  Measures 
(TQM/MGEEM)  is  part  of  the  Armstrong  Laboratory's  program  to 
provide  tools  and^  technologies  to  measure  and  enhance 
organizational  quality  and  effectiveness.  TQM/MGEEM  is  a 
significant  breakthrough  in  quality  measurement  which  provides  a 
powerful  set  of  new^  tools  for  improved  leadership  and  management 
and  a  means  of  periodically  soliciting  worker  input  to  identifv 
barriers  to  performance. 


T^^is  is  the  second  in  a  series  of  four  special  reports  written 
to  document  TQM/MGEEM.  The  first  is  intended  for  commanders  and 
leaders  ^  ^  of  ^  organizations  and  describes  their  primary 
responsibilities  and  what  is  necessary  to  start  a  TQM/MGEEM  effort. 
The  purpose  of  this  report  is  to  provide  TQM  facilitators  with  a 
step-by-step  guide  to  the  measurement  system  development  process 
that  is  an  integral  part  of  TQM/MGEEM.  The  third  report  explains 
to  leadership  and  facilitators  how  a  TQM/MGEEM  measurement  system 
IS  used  to  institute  and  cultivate  a  climate  of  continual 
improvement  and  how  Process  Improvement  Teams  (PITs)  fit  into  the 
TQM/MGEEM  picture.  The  fourth  report  is  intended  as  a  general 
work  ^  for  both  leadership  and  facilitators,  providing 
additional  details  on  numerous  aspects  of  the  TQM/MGEEM  technology, 
TQM  and  measurement  philosophy,  and  several  of  the  techniques  found 
in  the  TQM/MGEEM  system. 

The  authors  thank  Mr.  Larry  T.  Looper  for  his  invaluable 
ombudsmanship  in  support  of  this  research.  We  also  thank  the 
scores  of  people  at  conferences,  presentations  and  test  sites  that 
have  provided  valuable  insight  and  feedback  toward  the  continual 
improvement  of  both  TQM/MGEEM  and  our  presentation  of  it. 
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Implementing  Total  Quality  Management  (TQM)  II: 

A  Facilitators  Guide 

Introduction 

This  special  report  is  the  second  in  the  1992  series  of  guides 
on  implementing  Total  Quality  Management  (TQM)  with  the  Methodology 
for  Generating  Efficiency  and  Effectiveness  Measures  (TQM/MGEEM) . 
As  the  technology  first  documented  in  earlier  technical  papers 
(Tuttle  &  Weaver  1986,  Weaver  &  Looper  1989)  has  continued  to 
improve,  the  need  for  revised  and  updated  documentation  has  become 
^PP^rent,  resulting  in  this  series.  This  is  a  step— by— step  guide 
facilitators  in  developing  a  unique  measurement  system 
that  will  allow  a  target  organization  to  better  implement  TQM. 

The  first  in  this  series,  subtitled  "The  Command  Imperative,” 
addresses  the  role  of  top  management  in  implementing  TQM  and  is 
intended  primarily  for  the  leadership  in  a  target  organization 
(Weaver  &  Upton,  1992a) .  The  third  of  this  series  discusses  the 
role  of  a  TQM/MGEEM  measurement  system  and  feedback  meetings  in  the 
process  of  continual  improvement  and  is  intended  for  use  by  TQM 
facilitators  and  the  members  of  the  feedback  team  (Weaver  &  Upton, 
1992b).  The  fourth  of  this  series  is  a  general  reference  work 
providing  more  detailed  explanations  of  TQM/MGEEM,  how  it  relates 
to  TQM,  and  other  topics  (Weaver,  Upton  &  Frank,  1992).  It  is  a 
valuable  reference  for  all  personnel  involved  in  a  TQM/MGEEM 
effort.  ' 


Preparation 

It  is  assumed  that  the  readers  are  familiar  with  the  terms  and 
Ideals  of  the  philosophy  generally  known  in  the  Air  Force  as  Total 
Quality  Management  (TQM) .  The  Navy  refers  to  it  as  Total  Quality 
Leadership,  while  it  is  often  referred  to  Total  Quality  in  Japan 
and  among  much  of  corporate  America.  If  such  is  not  the  case  and 
the  reader  needs  knowledge  of  TQM  in  general,  several  books  provide 
a  good  introduction  to  the  topic.  The  first  we  generally  recommend 
IS  one  of  the  two  books  on  the  Deming  philosophy  written  by  Mary 
Walton,  The  Deming  Management  Method  (1986)  and  Demina  Management 
at  Work  (1990) .  A  good  follow  on  to  one  of  these  two  is  the  book 
ills — Deming — Guide — to  Quality  and  Competitive  Position  (1987)  by 
Howard  and  Shelly  Gitlow  and  The  Demina  Route  to  Quality  (1988)  by 
William  Scherkenbach .  When  the  learner  has  a  basic  grasp  of  the 
concepts  involved  they  are  ready  to  tackle  the  more  challenging 
presentation  found  in  Dr.  Deming's  own  Out  of  the  Crisis  (1986). 
Also  helpful  is  the  book  Juran  on  Leadership  for  Quality  (1989)  by 
Dr  Joseph  M.  Juran  which  explains  his  contributions  to  the  body  of 
TQM  knowledge  and  philosophy,  with  this  knowledge  one  is  ready  to 
embark  on  a  TQM/MGEEM  journey. 
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Preparing  the  Organization 


Implementing  TQM/MGEEM  in  an  organization  will  not  "install” 
TQM.  To  the  contrary,  using  TQM/MGEEM  in  an  organization  that  is 
not  implementing  the  Deming  philosophy  (Deming,  1986)  will  only 
result  in  the  imposition  of  yet  another  tool  for  micro-management; 
to  be  gamed,  avoided,  tricked  and  ultimately  scrapped  as  an 
impediment  to  productivity  and  quality.  Therefore,  the  establish- 
of  the  quality  councils,  studying  the  teaching  of  Dr  Deming 
and  the^ institution  of  training  in  the  area  of  TQM  are  necessary 
preconditions  to  successful  use  of  TQM/MGEEM.  The  reader  is 
referred  to  Weaver  &  Upton  (1992a)  for  more  details  on  leadership's 
obligations  in  implementing  TQM. 

Preparing  for  the  Implementation  of  TQM/MGEEM 

Out  of  the  Crisis.  Dr  Deming  tells  his  readers  that  they 
need  "other  knowledge"  to  help  toward  improvement  (Deming,  1986) . 
A  facilitator's  job  is  to  provide  this  "other  knowledge."  Toward 
this  end  a  ^  facilitator  must  take  several  steps  to  prepare  for 
implementation  of  TQM/MGEEM.  The  first  is  for  the  facilitator  to 
gain  rudimentary  knowledge  of  what  the  target  organization  does  and 
how  it  is  structured.  If  the  facilitator  is  a  member  of  the  target 
®^9®nization,  the  knowledge  he/she  already  has  is  usually  suffi¬ 
cient.  If,  however,  the  facilitator  is  from  outside  the  organiza¬ 
tion,  especially  if  he  or  she  is  from  a  dissimilar  work  environ¬ 
ment,  some  basic  familiarization  is  necessary.  This  does  not  mean 
facilitators  need  to  understand  exactly  what  the  target  organiza¬ 
tion  does  and  how  it  does  it.  Such  a  deep  understanding  implies 
that  the  facilitator  will  be  building  a  system  of  measurements, 
which  is  not  the  case.  The  facilitator's  role  is  one  of  moderator, 
not  architect,  a  role  of  scout  and  guide,  not  trail  boss  and 
leader.  The  facilitator  guides  teams  within  the  target  organiza¬ 
tion  through  a  process  by  which  organization  members  themselves 
build  a  system  that  will  serve  as  a  basis  for  improvement.  It  is 
important  to  understand  that  improvement,  not  measurement,  is  the 
ultimate  aim  of  a  TQM/MGEEM  implementation  effort.  A  review  of  the 
organizational  structure  chart  and  some  brief  discussion  with  the 
top  management  involved,  as  to  the  mission  of  the  organization, 
will  help  a  facilitator  better  understand  the  ideas  that  teams 
within  the  target  organization  will  be  shortly  presenting  to 
him/her. ^  This  will  bring  about  a  smoother  flow  of  ideas  and 
information  in  TQM/MGEEM  measurement  development  sessions. 

The  second  step  in  preparation  for  TQM/MGEEM  measurement 
development  is  to  identify  the  members  of  the  two  teams  that  will 
directly  participate  in  implementation.  Detailed  discussion  of  who 
should  be  on  each  team  is  found  in  the  following  sections  entitled 
"Players  on  the  Blue  Team"  and  "Players  on  the  Gold  Team".  Simply 
put,  the  Blue  Team  is  a  top  and  middle  management  team  and  the  Gold 
Team  is  a  middle  management  and  worker  level  team.  As  for  numbers 
of  participants,  suffice  it  to  say  that  each  team.  Blue  and  Gold, 
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should  consist  of  between  8  and  12  people  to  facilitate  the 
discussion.  More  than  12  participants  rapidly  degrades  the  ability 
groups  to  provide  a  smooth  flow  of  ideas,  unnecessarily 
lengthening  the  TQM/MGEEM  measurement  development  process.  Less 
than  8  may  damage  the  team's  ability  to  gather  the  needed  broad 
base  of  perspective  and  support  necessary  for  the  design  and 
implementation  of  a  rational  measurement  system. 

Sometimes  a  target  organization  is  sufficiently  small  to  place 
all  members  of  the  organization  and  the  selected  customer  and 
supplier  representatives  on  one  team  without  making  it  too 
unwieldy.  If  this  is  the  case  and  the  groups  desire  to  take  this 
course  then  by  all  means  proceed.  It  is  also  useful  to  designate 
one  of  the  team  members  as  the  recorder  for  the  team,  responsible 
for  keeping  a  record  of  the  various  documents  and  lists  developed. 
This  usually  takes  the  fojrm  of  one  of  the  team  members  taking  the 
various  working  documents  on  chart  paper  and  overheads  back  to  a 
secretary  and  having  them  typed  up,  copied  and  distributed.  Often 
the  commander  of  the  target  organization  will  volunteer  for  this 
duty  unasked  as  he  or  she  is  usually  eager  to  demonstrate  to  their 
subordinates  and  to  the  facilitator  their  commitment  to  the 
TQM/MGEEM  process  as  well  as  being  in  the  most  logical  position  to 
get  the  secretarial  support. 

After  identifying  the  members  of  the  Blue  and  Gold  Teams, 
sufficient  time  must  be  set  aside  for  conducting  the  development 
sessions.  There  are  no  hard  and  fast  rules  about  how  long 
TQM/MGEEM  takes,  but  some  rules  of  thumb  may  be  helpful.  First 
more  service-oriented  organizations  may  take  more  time  than 
production-oriented  ones  to  get  measures  of  what  the  organization 
does  even  though  it  has  been  shown  in  countless  service  organiza¬ 
tions  that  TQM/MGEEM  also  works  well  in  these  environments. 
Second,  the  less  specific  the  mission  of  the  target  organization, 
the  longer  the  Blue  and  Gold  Teams  will  take  to  accomplish  their 
tasks.  This  usually  applies  to  the  higher  echelons  in  an  organiza¬ 
tional  hierarchy  (e.g.  command  staff)  or  those  with  less  identifi- 
_ Products/ services  (e.g.  research  and  development  centers). 
Despite  their  difficulties  with  less  concrete  mission  statements, 
TQM/MGEEM  works  well  when  such  missions  have  been  clarified. 
Third,  the  more  people  involved  in  the  working  groups,  the  longer 
the  process  usually  takes.  This  is  why  the  size  of  the  groups 
should  usually  be  limited  to  about  a  dozen  participants.  With 
these  guidelines  in  mind,  TQM/MGEEM  has  taken  from  as  little  as  4 
hours  for  a  small  Security  Police  work-center  to  as  long  as  2  weeks 
for  the  headquarters  of  a  research  laboratory.  Usually,  however, 
about  8  to  12  hours  (over  2  or  3  days)  is  sufficient.  In  general, 
it  IS  suggested  that  the  Teams  set  aside  enough  consecutive  half 
more  hours,  i.e.  all  morning  or  all  afternoon)  to 
complete  their  tasks  without  significant  time  lapses  between 
sessions.  Usually  3  half-days  for  the  Blue  Team  are  sufficient 
while  2  half-days  for  every  3  KRAs  plus  a  half  day  for  the  weighing 
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table  and  Mission  Effectiveness  (ME)  charts  is  usually  enough  time 
for  the  Gold  Team  to  complete  their  tasks. 


The  choice  of  meeting  location  for  the  teams  can  immensely  aid 
a  successful  TQM/MGEEM  experience.  Often  it  is  helpful,  especially 
at  the  upper  echelons  of  an  organization,  to  hold  the  TQM/MGEEM 
sessions  at  a  physically  separate  site,  perhaps  even  in  a  different 
part  of  the  base  or  city  to  avoid  lost  time  due  to  "I'll  be  back 
from  my  office  in  a  minute,"  or  "urgent"  phone  calls  taking  key 
participants  away.  The  meeting  room  should  be  comfortable  and 
quiet  ^  as  participants  will  be  working  hard  on  building  a  system 
that  is  right  for  them.  There  should  be  plenty  of  room  for  each 
participant  to  spread  out  and  write,  and  the  room  should  be 
arranged  to  allow  everyone  to  see  both  the  overhead  projector 
screen  and  the  walls  where  chart  paper  with  ideas  written  on  it 
will  be  hung.  It  is  helpful  to  have  tables  arranged  in  a  "U"  shape 
with  the  opening  facing  the  front  of  the  room  and  the  participants 
seated  around  the  outside  edge  (see  Fig.  l) .  This  aids  the 

facilitator  in  maintaining  the  correct 
atmosphere  and  mindset  among  the  par¬ 
ticipants  and  allows  all  the  team  mem¬ 
bers  to  see  everything  with  a  minimum 
of  trouble.  Although  not  required  by 
anything  in  the  TQM/MGEEM  process  it¬ 
self.  This  facilitator  has  found  that 
refreshments  are  often  called  for  (at 
least  in  ^  a  good  natured  way) .  How  the 
leadership  of  the  target  organization 
wishes  to  handle  this  is  up  to  their 
discretion. 


P  P  P  p  p 


Finally,  some  basic  supplies  are 
needed  to  facilitate  the  TQM/MGEEM  ^  '  Facilitator 

implementation  process .  These  include  P  =  Participant 

an  overhead  projector  and  screen,  both 

for  pre-briefing  the  various  steps  and  *'i?nJre  1  Arrangement  of 
to  help  develop  ME  charts,  to  be  de-  Members, 

scribed  later  in  the  last  step  in  the 

process.  An  easel  with  a  supply  of  chart  paper,  along  with  markers 
for  writing  on  it,  should  also  be  available.  A  whiteboard  can  also 
be  very  useful  for  listing  the  ideas  of  the  participants  and  or 
working  on  the  mission  statement  with  the  Blue  Team.  Each  team 
member  should  have  about  50  3x5  index  cards  or  similar-sized  pieces 
of  paper  for  the  voting  portions  of  the  process,  as  well  as 
notepaper  and  writing  implements .  It  is  often  a  good  idea, 
especially  if  the  facilitator  is  from  outside  the  organization,  to 
provide  each  person  with  a  name  card  on  which  they  can  write  the 
name  they  wish  the  facilitator  to  use  when  calling  on  them.  Once 
all  the  preparation  is  finished,  TQM/MGEEM  system  development  can 
begin. 
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TQM/MGEEM  Steps  And  How  To  Take  Them 

a  discussion  of  each  of  the  steps  of  the 
TQM/MGEEM  implementation  process  and  the  participants  in  each  step. 
Included  under  each  heading  is  information  to  help  the  facilitator 
lead  the  two  working  ^  teams  to  consensus  on  the  products  of  each 
step.  Before  beginning  mission  statement  review  with  the  first 
team  (called  the  Blue  Team) ,  a  brief  outline  of  the  entire  process 
IS  often  helpful  to  orient  participants.  (See  "The  MGEEM  Process" 
Weaver,  Upton  &  Frank,  1992  or  use  the  outline  in  Appendix  A  of 
his  publication) •  When  the  second  (Gold)  team  meets  a  repeat  of 
this  outline  as  well  as  a  summary  of  the  results  of  the  steps  thus 
far,  IS  useful.  This  summary  of  results  is  best  accomplished  with 
the  active  help  of  the  overlap  members  of  the  Gold  Team  (i.e.  the 
members  that  were  also  on  the  Blue  Team) . 

Mindset  and  Potential  Pitfalls 


The  facilitator  should  inform  both  groups  before  the  start  of 
heir  efforts  of  the  mindset  that  will  make  the  deliberations  more 
fruitful.  The  members  of  each  team  should  play  "boss  for  a  day  " 
taking  the  viewpoint  of  the  commander  or  manager  of  the  organiza- 
lon  throughout  the  process.  As  much  as  possible  the  team  members, 
in  keeping  with  Dr  Deming's  point  #9  (optimize  team  efforts  toward 
the  aims  of  the  entire  organization) ,  (Doming,  1986)  should  take 
the  perspective  of  the  whole  organization  rather  than  the  more 
parochial  perspective  of  their  particular  part  of  the  organization. 
The  facilitator  should  encourage  each  team  member  to  listen  to  both 
facilitator's  instructions  and  the  ideas  and  thoughts  expressed 
by  their  fellow  team  members.  The  reason  for  some  of  the  actions 
they  will  take  will  not  be  clear  at  the  time  they  take  them, 
encourage  them  to  not  jump  ahead  but  to  trust  the  process.  The 
TQM/MGEEM  measurement  development  process  has  been  tested  in  a 
myriad  of  organizations  and  when  accompanied  by  a  true  commitment 
to  TQM  has  always  achieved  superb  results.  If  team  members  put 
their  trust  into  the  process,  chances  are  good  that  they  will  be 
more  than  happy  with  the  result. 

There  are  some  potential  pitfalls  that  the  facilitator  should 
be  aware  of  and  seek  to  avoid.  One  of  the  most  potentially  harmful 
for  the  group  and  the  most  emotionally  draining  for  the  facilitator 
to  avoid  IS  too  much  facilitator  influence.  There  is  a  fine  line 
to  tread  here.  On  the  one  hand  the  facilitator's  job  is  to  keep 
the  process  moving  forward  at  a  reasonable  pace  and  attempt  to 
steer  the  participants  away  from  "rabbit  chasing."  On  the  other 
hand,  if  the  facilitator  attempts  to  exert  too  much  control, 
especially  if  the  facilitator  suggests  changes  in  the  system,  the 
group  will  tend  to  lose  their  sense  of  ownership  in  the  system, 
thereby  defeating  the  purpose  of  this  process.  Unfortunately 
facilitators  tend  to  be  among  the  strongest  proponents  of  TQM  in  an 
organization  as  well  as  being  rather  extroverted  and  willing  to 
voice  an  opinion.  For  a  facilitator  to  stifle  such  natural  urges 
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to  "tell  them  what's  right”  is  emotionally  draining  but  absolutely 
necessary  for  the  success  of  the  teams  efforts. 

Another  pitfall  is  failing  to  properly  control  one  or  more 
disruptive  members  of  a  team.  If  a  team  member  uses  his  or  her 
position  or  rank  or  just  irascibility  to  dominate  the  discussion 
the  process  will  tend  to  be  needlessly  drawn  out  and  stressful  and 
potentially  valuable  ideas  from  other  team  members  may  be  lost.  If 
such  a  situation  arises  the  facilitator  should  rely  on  the  "rules 
of  engagement,”  especially  those  dealing  with  discussion  and  voting 
during  the  Nominal  Group  Technique  (NGT)  to  curb  the  disruptive 
team  member.  A  very  useful  part  of  the  NGT  is  the  rule  that  all 
discussion  is  directed  to  the  facilitator  and  the  facilitator 
recognizes  who  has  the  floor.  If  team  members  begin  to  speak 
directly  to  one  another,  a  simple,  "To  me,  please”  is  often 
sufficient  reminder.  A  final  option  is  for  the  facilitator  to  take 
the  person  aside,  say  at  a  break,  and  explain  that  their  behavior 
is,  in  the  facilitator's  judgement,  impeding  the  process.  If 
necessary  the  facilitator  may  later  add  that  they  disruptive  member 
may  produce  a  discontinuation  of  the  effort.  This  strategy  is  only 
effective  if  the  facilitator  has  the  support  of  the  superior, 
something  that  should  be  ensured  as  a  matter  of  course  before  the 
process  begins. 

Players  on  the  Blue  Team 

The  Blue  Team  usually  consists  of  the  commander  of  the  target 
organization,  the  commander's  immediate  superior,  the  commander's 
immediate  subordinates,  and  some  representative  customers  and 
suppliers. ^  Having  the  commander's  superior  on  the  Blue  Team 
provides  linkage  with  the  next  higher  level  in  the  organization. 
Having  the  commander's  immediate  subordinates  as  team  members 
provides  for  participation  by  "middle”  management,  and  the 
customers  and  suppliers  can  provide  valuable  insights  into  customer 
expectations  and  supplier  capabilities.  The  non-organizational 
members  all  can  provide  valuable  insights  as  the  organizational 
members  examine  and  explain  to  outsiders  what  the  organization 
does. 


The  membership  of  the  Blue  Team  is  defined  in  relation  to  the 
target  organization  and  the  definition  itself  is  open  to  alteration 
the  organizations  members.  For  each  target  organization  the 
make  up  may  vary  in  a  number  of  ways  (e.g.,  the  number  of  customers 
&  suppliers,  the  presence  or  absence  of  the  Boss's  boss,  the 
presence  of  non-direct  subordinates,  etc.).  Furthermore,  if  a  large 
organization  is  using  TQM/MGEEM  throughout  its  various  levels  and 
divisions,  the  same  person  may  serve  as  a  member  on  several 
different  Blue  Teams  and  as  a  member  on  the  Gold  Team  for  yet 
another  level  of  the  organization.  The  membership  is  all  relative 
to  the  target  organization.  For  example,  consider  the  partial 
wiring  diagram  for  a  fictitious  organization  in  Figure  2.  In  this 
example,  LtCol  Gomez  is  the  Commander  or  Manager  of  the  target 
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organization,  his  immediate 
superior,  Col  Brown,  is  also  a 
member  of  the  Blue  Team.  LtCol 
Gomez's  three  immediate  subordi¬ 
nates  (CMSgt  Jones,  Capt  Perry 
and  Maj  Lee)  sit  on  the  Blue 
Team  and  four  representative 
Customers  and/or  Suppliers  (Lt. 

Kenny,  Capt  Wilson,  MSgt  Fite 
and  Ms  Carlson)  round  out  this 
Blue  Team's  membership.  If  a 
Blue  Team  is  built  at  the  next 
lower  level  of  the  organization, 
say  in  the  target  organization 
supervised  by  Capt  Perry,  Capt 
Perry  would  serve  on  two  differ¬ 
ent  Blue  Teams  (one  as  immediate  subordinate,  and  one  as  Command¬ 
er/Manager)  .  It^ is  even  possible  that  Capt  Perry  might  serve  on 
one  or  more  additional  Blue  Teams  as  a  representative  customer 
and/or  supplier. 

Actions  of  the  Blue  Team 

B^view — the — Mission _ Statement .  Dr.  Deming's  most  recent 

statement  of  his  first  point  (1990  version)  encourages  organiza¬ 
tional  leaders  to  "Create  and  publish  a  statement  of  the  aims  and 
purposes  of  the  organization.  Management  must  demonstrate  to  all 
employees  their  constant  commitment  to  this  statement."  This 
indicates  that  developing  a  good  mission  statement  and  management's 
visible  commitment  to  it  is  essential  to  the  successful  implementa¬ 
tion  of  TQM.  To  this  end,  the  first  step  in  TQM/MGEEM  is  for  the 
Blue  Team  to  review  the  target  organization's  mission  statement. 
Unfortunately,  many  organizations  have  insufficient  and  outdated 
mission  statements,  and  often  cannot  even  locate  this  valuable 
d«pcument.  This  step  corrects  this  problem.  Using  either  an  old 
mission  statement,  or  a  "straw  man"  statement  proposed  by  one  of 
the  participants,  the  Blue  Team  discusses  what  the  organization 
does  and  how  to  formulate  what  it  does  into  an  understandable 
statement  that  everyone  in  the  organization  will  be  able  to  support 
and  draw  meaning  from. 

The  facilitator  should  open  the  mission  statement  review 
process  with  ^  some  example  mission  statements.  Examples  from 
similar  organizations  at  other  locations,  sister  organizations  in 
other  services ,  and  mission  statements  from  higher  levels  in  the 
organization's  chain  of  command  are  all  potentially  useful  inputs 
to  the  Blue  Team.  In  addition,  some  brief  training  on  the  purposes 
of  the  mission  statement  (see  Gitlow  &  Gitlow,  1987)  mav  be 
appropriate.  ^ 

It  is  helpful  if  there  is  some  document  for  the  Blue  Team  to 
start  with.  This  can  be  an  old  mission  statement  taken  from  the 


Figure  2  Example  Blue  Team  Mem¬ 
bership  . 


7 


organizational  chart  book,  or  one  of  the  examples  mentioned 
previously.  It  could  also  be  something  made  up  specifically  for 
this  occasion  by  one  of  the  members  of  the  Blue  Team.  If  the 
latter,  it  should  be  emphasized  that  this  is  a  ''strawman”  and  will 
probably  have  extensive  surgery  done  to  it  before  it  is  acceptable 
to  the  Blue  Team.  This  will  avoid  hurt  feelings  and  the  acceptance 
of  a  less  than  optimal  statement  to  avoid  them.  The  mission 
statement  is  too  important  for  one  person's  ego,  no  matter  who  that 
person  may  be,  to  get  in  the  way  of  a  quality  product.  The  mission 
statement  is  the  cornerstone  upon  which  the  rest  of  the  TQM/MGEEM 
process  and  indeed  the  activities  of  the  organization  rest.  It 
flows,  after  all,  from  Dr  Deming's  first  point.  Facilitators 
benefit  enormously  in  their  TQM/MGEEM  work  from  gaining  an 
understanding  Dr  Deming's  philosophy  before  starting  work  with 
their  first  Blue  Team. 

The  starting  mission  statement  should  be  displayed  on  an 
overhead  projector  or  whiteboard  so  that  changes  can  be  made 
directly  to  the  document  and  evaluated  and  discussed  by  the  entire 
Blue  Team.  When  displaying  the  starting  statement  on  the  overhead 
^  the  facilitator  should  leave  space  for  corrections  and 
modifications.  Often  the  new  mission  statement  will  bear  only  a 
slight  resemblance  to  the  original  one  due  to  the  number  of 
®®*p^i^ications  made  by  the  Blue  Team.  Give  this  step  sufficient 
time  and  effort  ~  as  mentioned  before  this  document  is  the 
cornerstone  of  the  TQM/MGEEM  process.  If  the  Blue  Team  members  are 
reluctant  to  begin  this  process  because  of  fear  or  a  feeling  of 
unease,  a  structured  idea  generation  process  such  as  the  NGT 
described  in  Weaver,  Upton,  &  Frank  (1992),  can  be  used  to  develop 
thoughts  or  statements  for  the  mission  statement  and  these  can  be 
®^^ung  together  in  some  manner  to  serve  as  the  mission  statement. 
This  variant  of  the  procedure  bears  a  great  resemblance  to 
developing  KRAs  and  using  them  to  build  a  mission  statement. 
Example  Mission  Statements  as  well  as  further  thoughts  on  the 
Mission  Statement  can  be  found  in  TQM  IV. 

Identify  and  Prioritize  Customers  and  Suppliers.  After  the 
mission  statement  has  been  discussed,  clarified  and  agreed  to,  or 
perhaps  as  part  of  that  process  (e.g.  to  sharpen  the  customer  focus 
that  a  good  mission  statement  should  have,  Gitlow  &  Gitlow,  1987) , 

Blue  Team  should  identify  and  prioritize  customers  and 
suppliers.  In  this  step  the  team  will  get  its  first  introduction 
to  the  use  of  the  NGT  (described  in  Weaver,  Upton,  &  Frank,  1992) 
along  with  the  first  of  the  two  NGT  voting  strategies  recommended 
in  TQM/MGEEM.  It  is  helpful  before  this  step  begins  for  the  group 
to  understand  two  ideas.  One  is  the  NGT  process  that  will  be  used 
and  the  other  is  the  systems  model  of  organizations.  (See  "The 
Nominal  Group  Technique,”  "Basic  System  (Input-Output)  Model”  and 
"Performance  and  Measurement”  in  Weaver,  Upton,  &  Frank,  1992  for 
a  discussion  of  these  items) . 
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The  facilitator's  introduction  of  the  ideas  of  customers  being 
organizations  or  persons  affected  by  an  organizations  output, 
that  the  outputs  are  information  and  services  as  well  as  products, 
etc.  should  conclude  with  definitions  of  customers  and  suppliers. 
Customers  and  suppliers  will  be  called  upon  in  the  future  to 
provide  inputs  to  improve  processes  within  the  target  organization 
and,  therefore,  should  be  identified  with  a  particular  person  or 
persons  or  at  least  a  specific  office  symbol.  Customer  and 
supplier  identifications  that  are  too  nebulous,  such  as  "the 
nation"  or  even  "the  Army"  or  "the  Air  Force,"  should  be  discour¬ 
aged.  Facilitators  are  urged  to  study  the  work  of  Dr.  Juran  on 
customers  and  suppliers  (Juran,  1989).  This  work  includes 
definitions  of  customers  and  suppliers,  knowledge  that  customers 
and  suppliers  are  internal  as  well  as  external,  and  the  famous 
triple  role  (TRIPROL)  concept.  His  teachings  on  the  triple  role  of 
orpnizations  (as  processor,  customer  and  supplier),  internal  and 
external  customers  and  his  definition  of  a  customer  as  anyone  who 
receives  or  is  affected  by  an  organizations  output  should  be  made 
clear  to  the  Blue  Team. 


During  the  customer  and 
supplier  identification  process 
participants  often  note  that 
although  some  suggested  custom¬ 
ers  are  definite  customers  and 
some  suggested  suppliers  are 
definite  suppliers,  there  are 
often ^  suggested  customers  and 
suppliers  that  can  be  grouped  as 

both  suppliers  and  customers . .  vaxuuoie 

nsight  especially  if  arrived  at  by  the  Blue  Team  members  them¬ 
selves  and  not  explained  to  them  by  the  facilitator.  This  insight 
is  correct  and  reinforces  the  importance  of  the  two-way  feedback 
arrows  in  the  systems  diagram  (Fig  3).  Knowledge  of  and  apprecia- 
tion  for  customers  and  suppliers  is  an  important  first  step  on  the 
road  toward  sensitivity  and  eventual  active  solicitation  of 
customer  feedback  and  pursuit  of  more  harmonious  supplier  relation- 
ships.  If  by  the  end  of  the  supplier  identification  process  the 
Blue  Team  has  not  had  this  insight,  it  should  be  given  to  them  by 
the  facilitator. 


This  is  a  potentially  valuable 


begin ^  the  customer  identification  process  each  person 
should  be  provided  with  a  copy  of  the  new  or  revised  organizational 
mission  statement,  or  the  mission  statement  should  be  placed  where 

facilitator  should  then  pose  the  customer 
Identification  nominal  question,  "Who  are  the  customers  of 
[organization  name]?"  Alternately,  the  question  can  be  worded  in 
accordance  to  Dr  Juran's  ideas  on  customers  (Juran,  1989)  as  "Who 
receives  the  outputs  of  [organization  name]?" 

When  the  voting  phase  of  the  NGT  is  reached,  participants  are 
instructed  to  vote  on  the  customers  they  think  are  most  critical. 
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It  is  understood  in  the  TQM  context  that  all  customers  are 
important/  but  a  relatively  small  percentage  of  them  are  critical 
to  the  success  of  the  organization.  This  is  based  on  the  Pareto 
principle  as  it  applies  to  customers.  This  principle  states  that 
a  relatively  small  portion  of  a  group  is  responsible  for  the 
majority  of  the  importance  of  that  group  (i.e.  20%  of  the  organiza¬ 
tion's  customers  are  critical  because  they  consume  80%  of  the 
organizations  output) . 

The  number  of  customers  each  participant  may  vote  for  is 
dependent  on  the  number  of  customer  names  left  after  the  discussion 
and  clarification  phase.  A  rule-of-thumb  is  to  allow  each 
participant  to  select  between  five  and  seven  "critical  customers." 
The  number  the  facilitator  chooses ,  whether  three,  five,  seven  or 
some  other  number,  is  left  to  the  facilitator's  discretion  and  the 
characteristics  of  the  voting  technique  used,  but  it  is  suggested 
^hat  no  more  than  half  the  total  number  of  customers  be  given  as 
the  number  of  votes  for  each  person.  Participants  write  the  names 
and  identifying  number  of  the  selected  customers,  one  per  card,  on 
3x5  index  cards  or  similarly  sized  pieces  of 
scratch  paper.  The  cards  are  turned  in  and 
participants  take  a  short  break  while  the 
votes  are  tallied  by  the  facilitator. 

To  report  the  results  of  the  voting  the 
facilitator  should  make  a  Pareto  chart  of  the 
results  to  identify  the  "critical  few"  cus¬ 
tomers  as  opposed  to  the  "important  many". 

(A  sample  vote  is  found  in  Fig  4,  the  result¬ 
ing  Pareto  chart  in  Fig  5) .  It  is  these 
critical  few  customers  with  whom 
the  organization  needs  to  build 
strong  contacts  in  order  to  define 
and  improve  the  quality  of  their 
operations . 

The  facilitator  then  repeats  « 
the  same  identification  process  , 
for  suppliers.  Facilitators  ‘ 

should  bear  in  mind  that  the  pur¬ 
pose  of  identifying  and  prioritiz¬ 
ing  is  twofold.  First,  to  provide 
the  organization  with  a  more  de¬ 
finitive  list  of  customers  and 
some  idea  of  their  relative  impor-  Figure  5  Pareto  Chart  of  Vot- 
tance,  and  secondly  to  begin  to  1*^9  Results, 
encourage  people  to  start  thinking 

in  a  more  holistic  manner,  considering  their  customers'  needs  and 
suppliers'  capabilities  in  their  day-to-day  activities.  Any  list 
of  critical  and  important  customers  and  suppliers  that  affects  the 
attitude  change  toward  the  teachings  of  TQM  and  provides  a  starting 
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point  for  building  relationships  with  customers  and  suppliers  is  a 
valuable  list. 

Identify  Key  Result  Areas.  The  next  step  in  the  TQM/MGEEM 
process  and  the  last  step  for  the  Blue  Team  is  to  identify  Key 
Result  Areas  (KRAs) .  KRAs  are  the  component  parts  of  the  mission 
statement,  the  small,  measurable  pieces  of  the  job  the  Air  Force 
pays  the  target  organization  to  accomplish.  The  facilitator  should 
explain  that  the  best  KRAs  are  the  make  or  break  portions  of  the 
mission:  readiness  for  a  flying  wing,  customer  service  for  a  supply 
function,  timeliness  for  a  service  organization.  KRAs  should  be  a 
realization  of  customer  expectations,  not  a  process,  tool,  or 
method  to  achieve  those  results.  A  copy  of  the  revised  mission 
®^^tement  for  each  participant,  or  displaying  the  revised  mission 
statement  either  on  a  whiteboard,  chart  paper,  or  overhead 
projector  can  serve  as  a  valuable  touchstone  for  the  Blue  Team 
members.  Some  examples  of  KRAs  can  be  found  in  Weaver,  Upton,  & 
Frank  (1992). 

The  NGT  is  again  used  to  identify  KRAs.  The  Blue  Team 
generates  answers  to  the  following  nominal  question:  "What 
categories  of  results  is  this  organization  expected  to  accomplish?” 
or  more  simply  "What  does  the  Air  Force  pay  [organization  name]  to 
do?”  After  silent  generation  and  round-robin  listing,  the  Blue 
Team  may  have  a  plethora  of  potential  KRAs.  Some  will  be  synonyms 
or  close  synonyms  and  are  often  combined  in  the  discussion  and 
clarification  phase.  Other  suggested  KRAs  will  not  be  viewed  by 
the  Blue  Team  as  critical  and  will  drop  out,  either  in  the  discus¬ 
sion  phase  or,  more  probably,  in  voting.  The  Blue  Team  should  be 
assured  that  voting  will  take  care  of  non-critical  KRAs,  and  that 
discussion  and  clarification  is  usually  not  used  to  eliminate  all 
of  them.  While  listing  KRAs,  plenty  of  space  should  be  left 
between  each  one  to  allow  for  wording  changes  during  the  discussion 
and  clarification  phase  to  follow.  Occasionally  it  is  useful  after 
the  discussion  and  clarification  phase  to  go  around  the  table 
asking  each  member  of  the  Blue  Team  to  choose  their  top  KRA  to 
advocate.  The  participant  then  chooses  a  KRA  and  tells  the  team 
why  he/she  thinks  that  KRA  is  the  most  important.  More  than  one 
person  may  choose  the  same  KRA.  It  is  suggested  that  the  supervi¬ 
sors  of  the  target  organization  go  last  so  that  he/she  will  not 
unduly  influence  the  expression  of  ideas.  This  option  should  be 
used  judiciously,  however.  If  there  is  fear,  or  reason  for  fear, 
in  an  organization,  this  will  cause  more  distress  than  it  is  worth, 
but  when  the  team  is  working  well  together  this  discussion  often 
eliminates  the  need  for  voting  altogether. 

If  all  the  excess  KRAs  are  not  eliminated,  voting  follows. 
Again  any  reasonable  voting  technique  may  be  used.  The  example 
that  follows  uses  the  technique  we  personally  prefer  but  if  you  or 
your  group  feel  more  comfortable  with  another  voting  technique,  use 
it.  In  our  example  a  Blue  Team  with  six  members  arrives  at  a  list 
of  12  potential  KRAs  after  discussion  and  clarification.  The 
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facilitator  gives  each  team 
member  five  index  cards  to 
serve  as  voting  slips  during 
the  voting  phase.  Each  team 
member  then  writes  the  name 
of  the  five  most  important 
KRAs  in  the  middle  of  the 
slips,  one  KRA  per  slip  (see 
Figure  6) .  As  a  check  to 
insure  that  the  votes  are 
tabulated  correctly,  they 
should  also  write  the  number 
designator  of  the  KRA  from 
the  chart  paper  lists  on  the 
walls  in  the  upper  left  hand 
corner  of  each  card. 

After  all  participants  have  chosen  their  five  most  important 
KRAs,  each  person  lays  all  five  cards  in  front  of  them  on  the  table 
where  all  the  choices  can  be  seen  at  once.  They  are  then  asked  by 
the  facilitator  the  following  guestion:  "If  you  could  have  only  one 
of  the  KRAs  before  you,  which  one  would  it  be?”  The  facilitator 
tells  the  participants  to  write  a  five  in  the  lower  right  hand 
corner  of  the  card  and  turn  it  over.  If  the  Blue  Team  is  not  using 
five  cards,  then  each  participant  would  give  the  first  card  chosen 
a  weight  equal  to  the  total  number  of  cards  each  person  has  been 
given  (e.g.  7  points  for  this  first  rated  card  of  seven  votes). 
Next  the  facilitator  asks,  "Of  the  remaining  choices,  if  you  had  to 
eliminate  the  least  important  one,  which  would  it  be?"  This  choice 
is  given  a  one  in  the  lower  right  hand  corner  and  turned  face  down. 
Next  the  facilitator  repeats  the  question,  "If  you  could  have  only 
one  of  the  remaining  KRAs  before  you,  which  one  would  it  be?"  This 
KRA  is  given  a  weight  of  four  in  the  lower  right  and  turned  over. 
Then  the  least  important  remaining  KRA  is  located  and  so  on  until 
all  the  KRA  choices  have  been  weighted.  The  facilitator  then 
collects  the  cards  from  all  participants.  Allow  another  short  (ten 
minute)  break  to  tabulate  the  results. 

Results  are  tabulated  in  a  manner  similar  to  the  tabulation  of 
customer  voting  but  with  added  information  of  the  weights  given 
each  vote  is  also  considered.  First,  separate  the  cards  according 
to  KRA  numbers  given  when  listed  on  chart  paper  (the  number  in  the 
upper  left  hand  corner).  All  votes  for  KRA  #1  are  placed  in  one 
pil®/  for  #2  placed  in  another  pile,  etc.  Second,  put  the 
cards  in  each  stack  in  order  according  to  the  weight  in  the  lower 
right-hand  corner  of  each  card.  Record  the  weight  on  a  tally  sheet 
as  shown  in  Figure  7.  In  Figure  7,  the  first  column  is  the  number 
of  the  KRA  in  question.  The  second  column  is  a  list,  in  ascending 
order,  of  the  weights  given  that  KRA  by  each  participant  who  voted 
for  it.  The  final  column  is  asummary  of  the  voting  information. 
It  is  expressed  as  the  sum  fo  the  weights  written  above  a  dividing 
line  and  the  number  of  votes  for  each  KRA  written  below  the  di- 
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viding  line  in  the  manner  of  a 
fraction. 

There  are  no  rigid  rules 
for  interpreting  voting  results. 

These  data  are  gathered  so  that 
the  Blue  Team  as  a  whole  will 
have  a  general  idea  of  the  rela¬ 
tive  importance  the  members 
attach  to  each  KRA.  The  goal  is 
to  reduce  the  number  of  KRAs 
down  to  a  parsimonious  system. 

Such  a  system  will  provide  a 
basis  for  improvement  without 
imposing  an  undue  burden  on 
those  being  measured  or  produc¬ 
ing  information  overload  for 
those  who  need  to  interpret  the  data.  Facilitators  are  urged  to 
read  Tom  Peters'  book  Thriving  on  Chaos  (1987) ,  especially  the  next 
to  last  chapter,  for  his  ideas  about  the  importance  of  making 
measurement  systems  simple,  parsimonious,  and  useful. 


1 

2,3 

5/2 

2 

4,5,5 

14/3 

3 

2,3 

5/2 

4 

4,5 

9/2 

5 

1,1, 1,1, 2, 2 

8/6 

6 

1,2,2 

5/3 

7 

1 

1/1 

8 

4, 4, 4, 4, 5, 5 

26/6 

9 

3 

3/1 

10 

3,3 

6/2 

11 

3 

3/1 

Figure  7  Recording  a  Sample  Key 
Result  Area  Vote 


A  number  of  KRAs  can  be  eliminated  as  a  result  of  the  vote. 
For  instance,  KRA  8  should  probably  be  included  in  the  system.  It 
received  a  vote  from  each  member  but  it  also  received  the 
highest  sum  of  weights  (26) .  Although  KRA  5  received  a  vote  from 
each  person,  all  of  them  apparently  thought  (assuming  from  the  weak 
weights)  it  had  relatively  little  value  as  compared  with  some  of 
K^s.  The  Blue  Team  may  therefore  decide  to  drop  KRA  5, 
although  this  is  not  required.  Although  KRA  2  received  only  three 
votes,  all  three  participants  who  voted  for  it  ranked  it  high,  with 
weights  of  4  or  5.  Dropping  this  KRA,  which  half  the  group  felt 
w^s  highly  important,  may  alienate  a  substantial  portion  of  the 
Blue  Team  and  later  a  similarly  substantial  portion  of  the 
organization.  Although  it  received  the  vote  of  only  half  of  the 
participants,  it  should  be  dropped  only  after  careful  consideration 
"y  Blue  Team.  Similar  comments  can  also  be  made  for  KRA  4 

(albeit  to  a  lesser  extent).  On  the  other  hand,  KRAs  7,  9,  and  li 
could  probably  be  dropped  with  little  impairment  to  either  the 
system  or  the  comfort  of  the  participants. 


If  consensus  on  a  small  set  of  KRAs  cannot  be  reached  after  a 
first  vote,  a  second  and  perhaps  third  vote  may  be  necessary, 
nulling  some  of  the  candidates  each  time.  Once  consensus  has  been 
reached  by  the  Blue  Team  they  are  thanked  for  their  efforts  and 
dismissed.  The  results  of  their  efforts  are  documented  by  the 
facilitator  or  the  prearranged  scribe  and  presented  in  writing  to 
the  Gold  Team  as  they  begin  the  work  on  the  next  part  of  the 
TQM/MGEEM  process. 


13 


The  members  of  the  Gold  Team  again  can  only  be  described 
relative  to  the  target  organization  in  question.  The  commander's 
immediate  subordinates,  who  were  also  on  the  Blue  Team,  and  key 
workers  (who  are  usually  among  the  immediate  subordinates  of  the 
commander  or  managers  immediate  subordinates)  make  up  the  member¬ 
ship  of  the  Gold  Team.  The  immediate  subordinates  provide  for 
linkage  between  the  two  teams,  and  both  they  and  the  key  workers 
are  knowledgeable  enough  to  build  measures  for  the  work  that  they 
do.  Key  workers  are  those  workers  who  by  virtue  of  their  position 
or  their  innate  charisma,  are  formal  or  informal  leaders  of  the 
work  force.  By  including  these  key  workers,  not  only  is  the 
viewpoint  of  the  workers  represented  effectively,  but  these  workers 
will  be  able  to  return  to  their  comrades  and  "sell”  them  on  the 
idea  of  using  the  system  since  these  work  force  leaders  were  part 
of  the  process  of  creating  the  measures.  An  example  based  on  the 
fictitious  organization  used  to  illustrate  the  blue  team  may  help 
(Fig  8) .  CMSgt  Perry,  Capt 
Jones  and  Maj  Lee  again  sit 
on  the  team  as  the  immediate 
subordinates .  The  represen¬ 
tative  customers  and  suppli¬ 
ers  are  gone  as  well  as  the 
senior  leadership  (Col  Brown 
and  LtCol  Gomez)  .  We  also 
have  key  workers  on  this  |_ 

team,  six  of  whom  are  the 

direct  subordinates  of  the  Figure  8  Example  makeup  of  the 
three  overlap  members.  The  Gold  Team, 
final  member  of  the  team 

(SSgt  Lewis)  is  also  a  key  worker,  but  is  not  a  direct  subordinate 
of  one  of  the  three  immediate  subordinates.  SSgt  Lewis  may  be  a 
key  worker  because  the  section  that  Capt  Allison  leads  is  large  and 
SSgt  Lewis  is  a  needed  "second  representative"  of  the  leadership  of 
that  section,  or  perhaps  SSgt  Lewis  is  seen  by  LtCol  Gomez,  et  al 
as  being  an  informal  spokesman  for  the  work  force — someone  who  is 
well-informed  about  their  concerns  and  can  better  relate  to  the 
"front-line"  workers. 

Actions  of  the  Gold  Team 

The  Gold  Team  is  responsible  for  developing  measures  for  the 
KRAs  developed  earlier  by  the  Blue  Team.  Again  a  briefing  on  the 
overall  TQM/MGEEM  process  would  be  helpful.  Also  helpful  would  be 
a  few  words  about  how  measurement  and  inspection  fit  into  the  TQM 
philosophy  (see  Dr  Deming's  point  #3,  etc.  (Deming,  1986)).  The 
goal  of  improvement  should  be  stressed  and  the  measures  need  only 
be  "good"  enough  to  provide  a  basis  for  that  improvement,  they  need 
not  be  overly  precise. 


.  Develop  Indicators  for  KRAs.  The  first  task  for  the  Gold  Team 
IS  to  develop  indicators  for  each  of  the  KRAs  identified  by  the 
Blue  Team.  Before  beginning  this  step  Gold  Team  members  who  were 
also  on  the  Blue  Team  (commander's  immediate  subordinates)  should 
informally  brief  the  other  members  of  the  Gold  Team  about  what  was 
accomplished  by  the  previous  team.  Each  member  of  the  Gold  Team 
should  have  a  copy  of  the  Blue  Team's  results,  including  the 
mission  statement,  critical  customers  and  suppliers,  and  a  list  of 
the  KRAs.  All  Gold  Team  member  questions  should  be  answered  by  the 
previous  Blue  Team  members. 


'Hard'  Measures 

\ 


A  few  words  on  three  kinds  of  measures  (Fig  9) ,  sources  of 
measurement  data  and  the  characteristics  of  a  good  indicator  that 
can  be  very  helpful  to  the  Gold  Team  members  at  this  time.  The 
facilitator  should  explain  to  the  Gold  Team  members  that  measures 
(indicators)  come  in  three  basic  types.  “Hard”  measures  (e.g. 
reject  rates,  on-time  take-offs, 
on-time  deliveries);  “soft," 
more  subjective  measures,  (e.g. 
customer  survey  ratings,  expert 
opinions)  and  infeasible  mea¬ 
sures,  or  measures  that  cost 
more  than  they  are  worth.  An 
example  of  an  infeasible  measure 
is  an  indicator  suggested  by  a 
civil  engineering  squadron  at 
one  base.  They  initially  sug¬ 
gested  that  a  survey  be  sent  to 
every  person  on  the  base  once  a 
month  and  that  this  multi-page 
survey  be  filled  out,  returned 
and  tabulated  to  form  a  part  of  Figure  9  Types  of  measures, 
their  indicator  system.  This 

noble  idea  was  discarded,  however,  when  it  became  obvious  that  a 
hour^  investment  would  have  to  be  made  every  month  to 
handle  the  dissemination,  collection  and  tabulation  of  the 
instri^ent,  not  to  mention  the  base— wide  investment  of  time  away 
from  immediate  work  to  complete  the  questionnaire. 
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'Not  Feasible' 

At  Reasonable  Cost 


'Soft'  Measures 


The  Gold  Team  should  understand  that  indicator  data  can  be 
gathered  in  a  wide  variety  of  ways.  Of  course  the  most  preferred 
way  IS  direct  measurement  of  the  work  processes  themselves,  either 
through  work  sampling,  efficiency  measures  of  varying  kinds,  and/or 
timing^  measurements.  ^  Another,  less  desirable  method  may  be 
gathering  formal  and  informal  feedback  from  customers  and  from 
members  of  the  organization  who  interface  with  external  customers. 
If  none  of  these  measures  are  available  or  obtainable  at  a 
reasonable  cost,  opinions  of  key  workers  or  members  of  management 
using  scales  of  various  types  can  be  used.  Any  of  these  sources  of 
data  can  be  used  by  the  Gold  Team  as  sources  for  their  indicators 
if  they  keep  in  mind  that  the  entire  goal  is  to  build  measures  that 
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will  serve  as  a  basis  for  improvement  of  the  organizations  mission 
execution . 

Experience  shows  that  good  indicators  share  several  common 
characteristics.  For  one,  they  are  easily  understood.  They  do  not 
require  extensive  calculation  or  explanation.  Good  indicators  are 
important  and  valid  to  leaders  and  workers  involved  in  evaluating 
them.  This  means  that  both  workers  and  leadership  believe  that  the 
measure  is  meaningful  (i.e.,  measuring  the  right  things).  Indica¬ 
tors  should  concentrate  on  outputs  controlled  by  the  target 
organization  and  not  by  outside  forces,  although  major  external 
inputs  might  be  tracked  in  keeping  with  Dr  Deming's  point  #4 
concerning  building  a  relationship  with  suppliers.  The  indicators 
should  reflect  efficiency  (doing  things  right,  output  divided  by 
input)  as  well  as  effectiveness  (doing  the  right  things,  meeting 
customer  needs) .  They  should  evaluate  change  by  being  responsive 
to  actions  taken  by  the  target  organization  to  improve  the 
Processes  which^  drive  them,  should  be  cost  efficient  by  using 
existing  or  easily  gathered  data,  and  do  not  need  to  be  measured 
with  extreme  precision.  Remember  that  measurement  is  not  of  key 
import,  improvement  is.  Measures  need  only  be  good  enough  to  serve 
as  a  basis  for  improvement.  More  information  on  measurement  in 
general  as  well  as  examples  of  indicators  can  be  found  in  Weaver, 
Upton,  &  Frank  (1992) . 

Indicators  are  developed  in  a  number  of  similar  indicator 
development  cycles  (one  cycle  for  each  KRA) .  To  generate  indica¬ 
tors  a  repeat  of  the  NGT  process  used  for  developing  the  KRAs  is 
used.  ^  In  this  session  the  nominal  question  for  each  cycle  is  "What 
quantitative  indicator (s)  should  the  manager  of  [organization  name] 
track  on  a  periodic  basis  to  tell  whether  the  KRA  is  being 
accomplished?"  For  each  KRA  previously  identified  by  the  Blue 
Team,  this  question  is  used  in  a  silent  generation  phase  of  the 
NGT.  After  going  through  all  the  steps  of  the  NGT  (from  posing 
nominal  question  to  vote  tabulation)  indicators  for  each  KRA  are 
generated . 

Each  indicator  should  have,  in  addition  to  it's  letter 
designation,  a  name  and  a  specification  determined  when  the 
indicator  is  first  suggested.  The  specification  should  include 
where  the  data  are  coming  from,  how  the  data  are  generated,  the 
time  period  over  which  the  data  are  tracked,  and  the  interpretation 
the  values,  i.e.,  whether  a  higher  number  better  or  worse. 
After  voting  has  identified  the  final  indicator  or  indicators  for 
a  KRA  the  facilitator  should  lead  the  Gold  Team  to  determine  a 
feasible  best,  a  feasible  worst  and  an  indifference  point  for  the 
indicator (s) .  Identifying  these  three  values  at  this  point  is 
important  because  the  reasoning  behind  their  generation  may  fade 
and  later  steps  may  be  rougher  because  of  the  values  absence. 

When  correctly  developed,  each  indicator  is  the  result  of  a 
process  important  to  fulfilling  the  mission  of  the  organization. 
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the  "head"  of  a  fishbone  diagram 
(Brassard,  1988)  such  as  shown 
in  Figure  10.  Therefore  the 
feasible  best  and  feasible  worst 
should  be  the  boundaries  of  the 
processes  capability.  The  fea¬ 
sible  best  is  the  best  the  tar¬ 
get  organization  can  reasonably 
expect  from  that  process  during 
the  reporting  period.  The  fea¬ 
sible  best  is  not  the  arithmetic 
or  absolute  best  but  the  best 
that  can  reasonably  be  expected 
if  everything  goes  right  in  the  Figure  10  Fishbone  to  Mission 
process  which  drives  the  indica-  Effectiveness  Chart, 
tor  in  question.  Similarly,  the 

feasible  worst  may  not  be  the  arithmetic  or  absolute  worst  but  only 
the  worst  the  target  organization  may  do  on  an  indicator  if 
everything  in  the  process  behind  the  indicator  that  can  reasonably 
go  wrong  does  go  wrong.  The  indifference  point,  sometimes  referred 
to  as  the  "don't  rock  the  boat"  point,  is  the  performance  level  on 
an  indicator  at  which  the  target  organization  is  neither  recognized 
for  superior  achievement  nor  criticized  for  lackluster  performance. 
This  is  the  point  at  which  the  indicator  neither  significantly  adds 
to  nor  detracts  from  mission  accomplishment.  This  indifference 
point  may  be  a  zone,  a  range  between  two  numbers  instead  of  an 
exact  number.  An  example  of  these  three  points  might  be  "percent 
of  on  time  take-offs"  where  a  flying  squadron's  historical  high  is 
99%  (less  than  the  absolute  high  of  100%),  a  historical  low  might 
be  93%,  and  the  indifference  point  (determined  by  the  headquarters 
to  be  the  command-wide  average)  may  be  96%.  The  group  may  instead 
decide  that  they  would  have  a  zone  between  95%  and  97%  as  their 
indifference  point.  This  may  be  a  standard  that  has  not  been 
identified  as  arbitrary  (in  violation  of  Dr.  Deming's  point  11)  or 
merely  the  reflection  of  the  consensus  of  the  team  as  to  the  policy 
of  their  organization. 

The  Gold  Team  should  now  put  the  indicator (s)  for  that  KRA 
aside  and  go  on  to  generate  indicators  for  the  next  KRA.  The  Gold 
Team  repeats  this  process  until  measures  are  generated  for  all  the 
KRAs  identified  by  the  Blue  Team.  Once  all  indicators  are 
identified,  and  only  then,  the  Gold  Team  is  ready  to  rank  the 
indicators — one  against  another — in  preparation  for  developing  ME 
Charts . 

The  Indicator  Weighing  Table.  One  of  the  most  powerful 
aspects  of  the  measurement  systems  developed  by  TQM/MGEEM  is  the 
way^  in  which  indicators  for  different  KRAs  can  be  compared  and 
decisions  made  about  resource  allocation  and  continual  improvement. 
(This  feedback  process  is  discussed  in  Weaver  &  Upton,  1992b.) 
This  requires  that  all  indicator  feasible  bests  and  feasible  worsts 
be  ranked.  This  ranking  is  accomplished  with  an  indicator  weighing 
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table  and  makes  possible  the  development  of  ME  Charts,  the  heart  of 
the  TQM/MGEEM  measurement  system. 


Feasible 

Worst 

Feasible  Best 

1  n  d  i  f  f . 

Point 

Num 

Rank 

BP 

Num 

Rank 

EP 

^‘rovjde  Software 

Programming  Efficiency 

200 

1000 

600-700 

T  {  m  e  1  i  n  e  s  5 

-1.5 

1  .  5 

0  .  5 

^rovicie  Computer  Services 

U  p  t  i  m  e 

90* 

98% 

■ 

97% 

Access  to  Software 

100 

10 

40-30 

Provide  numerical  processing 
Meet  customer  needs 

25% 

0% 

15% 

(1) 

[2) 

(3) 

(4) 

(5) 

(fi) 

{7) 

(8) 

Figure  11  Indicator  Weighing  Table. 


An  indicator  weighing  table  is  developed  by  writing  the  names 
of  all  the  indicators  for  all  the  KHAs  in  the  first  column  as  shown 
in  Figure  11.  (Note  that  the  numbers  for  the  feasible  bests, 
worsts,  and  indifference  points  in  the  table  are  examples;  it  is 
not  necessary  to  understand  the  specifics  of  how  this  fictitious 
Gold  Team  arrived  at  these  numbers  or  exactly  what  the  n^lmbers 
represent.)  The  next  three  columns  relate  to  the  feasible  worst 
for  the  indicators.  The  first  of  these  columns,  labeled  "Num” 
contains  the  "feasible  worst"  scores  that  were  determined  for  the 
indicators  at  the  end  of  each  indicator  identification  cycle  the 
other  two  of  these  columns,  labeled  "Rank"  and  "EP"  will  be 
explained  later.  Columns  5-7  are  the  same  as  columns  2-4,  but  are 
for  the  feasible  best  score.  The  last  column  (labeled  IP)  contains 
the  indifference  point  or  zone  for  the  indicator. 

After  the  facilitator  has  completed  the  table  to  the  point 
shown  in  Figure  11,  the  Gold  Team  is  ready  to  begin.  Concentrating 
first  on  the  feasible  best  columns,  the  facilitator  should  pose  the 
following  question  to  the  Team,  "If  everything  went  well  and  this 
organization  scored  its  feasible  best  on  all  of  its  indicators, 
which  score  or  scores  would  help  your  overall  performance  the 
most?"  The  indicator  whose  feasible  best  is  the  most  helpful 
across  all  of  the  KRAs  is  then  given  a  rank  of  1  in  column  6 
(Feasible  best  —  Rank) .  After  identifying  the  most  helpful,  the 
facilitator  should  ask,  "Of  the  indicators  remaining,  if  you  were 
to  score  the  feasible  best  on  all  of  them,  which  one  would  help 
your  overall  performance  the  most?"  This  indicator  receives  a  rank 
of  2.  This  process  is  repeated  until  all  the  indicator  feasible 
bests  have  been  ranked.  If  two  or  more  indicator  feasible  bests 
are  equally  helpful  to  performance,  then  those  indicators  receive 
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an  equal  ranking  (e.g.  “ Programming  Efficiency”  and  ’’Meet  Customer 
Needs”  in  the  example) . 

After  ranking  all  the  feasible  bests,  an  effectiveness  point 
value  is  assigned  to  each.  The  EP  value  for  the  rank  1  indicator 
IS  set  at  100  (column  7),  and  the  facilitator  asks  the  following 
question,  "If  the  feasible  best  (ranked  1)  is  worth  100  effective¬ 
ness  points,  how  many  points  is  the  feasible  best  ranked  second? 
If  it  is  almost  as  beneficial  it  would  get  99  or  95  effectiveness 
points.  If  it  is  half  as  beneficial  to  effectiveness  as  rank  1  it 
would  get  50  points,  etc.”  The  object  is  to  determine  how  many 
points  the  second  ranked  feasible  best  is  worth.  In  the  example 
(Fig  12)  the  second  ranked  feasible  best  is  assigned  90  effective¬ 
ness  points.  This  process  is  repeated  until  all  the  feasible  bests 
have  been  assigned  an  effectiveness  point  value.  Note  that  an 
indicator  with  a  lower  ranking  must  have  a  lower  effectiveness 
point  value,  even  if  it  is  only  one  point  less,  because  the  Team 
ranked  it  lower  in  impact  on  the  mission. 


Feasible  Worst 

Feasible  Best 

1  nd  1  f  f  . 

Point 

Num 

Rank 

EP 

Num 

Rank 

EP 

Provide  Software 

Programmi ng  Efficiency 

2Q0 

1000 

2 

90 

600-700 

T  i  m  e  1  i  n  e  s  s 

-1,5 

1  .  5 

1 

100 

0  .  5 

Provide  Computer  Services 
Uptime 

90% 

98% 

4 

20 

97% 

Access  to  Sof  tware 

100 

10 

3 

60 

40-30 

Provide  numerical  processing 
Meet  customer  needs 

25% 

0% 

2 

90 

15% 

(1) 

(2) 

(3) 

H) 

(S) 

(6) 

(7) 

(8) 

Figure  12  Weighing  Table  With  Feasible  Best  Columns  Completed. 


^  ^  This  process  is  repeated  for  the  feasible  worsts  for  each 

indicator  with  two  exceptions.  First,  negative  effectiveness  point 
values  are  assigned  to  the  rankings  instead  of  positive  ones  and, 
second,  the  feasible  worst  ranked  1  does  not  necessarily  have  to  be 
assigned  -100  points.  It's  a  question  for  the  Gold  Team  of  how 
much  damage  to  mission  effectiveness  the  first  ranked  feasible 
worst  can  do  relative  to  the  benefit  that  the  first  ranked  feasible 
best.  In  real  life,  these  can  be  symmetrical,  but  don't  have  to  be 
worth  100  points  positive  and  negative.  The  facilitator  should  not 
allow  the  Gold  Team  to  get  bogged  down  in  trying  to  get  an  exact 
f sctiveness  point  value  for  each  of  the  ranks.  The  difference 
between  an  EP  rating  of  90  and  95  is  not  important  enough  to  spend 
a  substantial  amount  of  time  on.  Remember  the  goal  is  improvement 
of  processes,  not  precise  or  hyperaccurate  measurement.  Also,  it 
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should  be  understand  that  the  indicator  ranking  of  feasible  worsts 
is  usually  not  a  mirror  image  of  the  ranking  of  the  feasible  bests. 
Often  certain  indicators,  when  done  well,  garner  little  added 
effectiveness  but  when  done  poorly  (i.e.  when  the  indicator 
approaches  the  feasible  worst)  then  everyone  begins  to  scream.  An 
example  of  this  would  be  "Uptime”  in  the  previous  three  figures. 
If  the  computer  is  down,  everyone  screams,  but  if  it  is  up  100 
percent  of  the  time  one  month,  nobody  notices.  An  example  of  the 
opposite  case  is,  unfortunately,  much  rarer,  but  one  possibility 
might  be  an  indicator  for  morale  (or  quality  work  force)  of 
"Awards".  If  nobody  is  getting  anything,  they  don't  know  to 
complain,  but  if  lots  of  awards  are  being  garnered  by  the  organiza¬ 
tion  members,  it  will  really  increase  morale.  A  completed 
Indicator  Weighing  table  is  shown  in  Figure  13. 


Feas  i  b  1  e  Worst 

Feasible  Best 

1  nd  i  f  f . 

Point 

Num 

Rank 

EP 

Num 

Rank 

EP 

='rovide  Software 

Programmi ng  Efficiency 

200 

5 

-70 

1000 

2 

90 

600-700 

Time! I n  e  s  s 

-1.5 

3 

-80 

1  .  5 

1 

100 

0  .  5 

Provide  Computer  Services 

Up  t  j  me 

90« 

1 

-  100 

SB% 

4 

20 

Bl% 

Access  to  Software 

100 

2 

-95 

10 

3 

60 

40-30 

Provide  numerical  processing 
Meet  c  u  s  t  ome  r  needs 

25« 

4 

-60 

Q% 

2 

90 

\^% 

(M 

{2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

Figure  13  Completed  Indicator  Weighing  Table. 


With  the  completed  Indicator  Weighing  Table  in  hand,  the  Gold 
Team  is  now  ready  to  build  their  ME  Charts. 

Developing  Mission  Effectiveness  Charts.  An  ME  Chart  is 

a  graph  upon  which  the  Gold  Team  draws  a  line  or  curve  reflecting 
policy  on  the  relationship  between  the  range  of  feasible  values  on 
an  indicator  and  the  impact  on  the  overall  mission  effectiveness  of 
the  target  organization.  Example  ME  Charts  based  on  the  first  two 
indicators  in  the  Indicator  Weighing  Table  shown  in  Figure  13  can 
be  found  in  Figure  14.  (Note  that  the  Armstrong  Laboratory  has 
developed  software  to  aid  in  the  construction  and  maintenance  of  ME 
charts  and  indicator  data  (Upton,  1990) ) . 

To  begin  the  development  of  an  ME  Chart,  the  facilitator 
should  take  a  blank  chart  (an  example  is  in  Appendix  B  from  which 
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a  transparency  can  be  made)  and 
draws  the  feasible  worst,  fea¬ 
sible  best  and  indifference 
points  previously  decided  upon 
by  the  Gold  Team  on  the  chart. 

Figure  15  shows  an  example  of 
this  for  the  "Uptime”  indicator 
from  the  Indicator  Weighing 
Table  in  Figure  13.  The  verti¬ 
cal  axis  of  ME  Charts  should 
always  run  from  -100  to  100 
unless  the  ME  points  for  feasi¬ 
ble  worst  are  a  larger  magni¬ 
tude  nvimber  than  -100  (Weaver  &  . 

Looper,  1989)  .  In  this  case  Figure  15  Starting  an  ME  Chart. 

the  facilitator  should  allow 

the  axis  to  extend  from  100  down  to  whatever  negative  value  is 
required  by  the  Gold  Team.  To  enhance  the  legibility  of  ME  Charts 
the  facilitator  should  leave  some  space  on  the  charts  to  the  left 
of  the  feasible  worst  (which  is  the  left-most  number  on  the 
horizontal  axis)  and  some  space  to  the  right  of  the  feasible  best 
(the  right-most  number  on  the  horizontal  axis) . 


If  the  Gold  Team  does  not  have  any  objection,  these  points  are 
then  connected  with  two  straight  lines  (e.g.  "Programming  Efficien¬ 
cy"  in  Fig  14) .  However,  Gold  Teams  may  believe  there  is  some 
reason  to  place  an  inflection  point  somewhere  between  points  or  to 
draw  a  curve  (function)  to  represent  the  relationship  between 
indicator  value  and  mission  effectiveness.  That  may  be  done  as 
well.  The  Gold  Team's  wishes  on  how  the  points  are  connected  must 
be  strictly  observed.  Continuing  with  the  example  from  Figure  15, 
it  may  be  determined  by  the  Gold  Team  that  "Uptime"  has  a  fairly 
shallow  slope  between  90%  and  95%,  but  a  steeper  slope  between  95% 
and  the  indifference  point  at  97%.  The  Gold  Team  may  decide  that 
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the  ME  points  at  95%  are  about  - 
80  and  use  straight  lines  to 
connect  the  points  thus  obtained 
(Fig  16)  .  Sometimes  the  curve  f 
may  take  other,  less  common  f 
forms.  More  examples  can  be  § 
found  in  Weaver,  Upton,  &  Frank  | 

(1992).  V 

e 

The  facilitator  repeats  e 
this  process  for  each  indicator  | 
until  all  indicators  have  an  ME 
Chart.  After  ME  Charts  are 
completed,  it  is  suggested  that 
the  Gold  Team  put  away  their 
charts  for  a  couple  of  days  and  Figure  16  Finished  ME  Chart, 
then  meet  again  to  ensure  the 

curves  are  correct.  Such  a  "sanity  check"  can  also  be  done  at  the 
time  of  the  first  feedback  session.  Later,  when  real  data  are 
available  after  one  measurement  cycle's  data  has  been  gathered, 
another  feedback  session  with  a  sanity  check  is  held. 

After  several  sanity  checks  the  TQM/MGEEM  measurement  system 
may  be  briefed  to  the  appropriate  level  of  management  above  the 
target  organization.  These  managers  should  have  studied  Dr. 
Deming's  (1986)  philosophy  and  understand  that  the  central  purpose 
of  TQM/MGEEM  is  to  foster  Dr.  Deming's  teachings,  especially  points 
3  and  5  on  inspection  and  continuous  improvement  of  processes . 
They  should  be  educated  to  resist  their  natural  impulse  to  use 
TQM/MGEEM  as  another  tool  for  micromanagment .  They  should 
understand  Dr.  Deming's  advice  that  such  action  is  the  road  to 
organizational  ruin. 

With  the  completion  of  the  ME  Charts,  the  formal  TQM/MGEEM  is 
now  complete.  This  does  not  mean  that  TQM  is  "installed"  and  now 
will  operate  automatically.  The  most  valuable  steps  in  the 
TQM/MGEEM  process  are  about  to  begin. 


Using  TQM/MGEEM  To  Maximum  Benefit 

It  should  be  understood  that  the  KRAs,  Indicators,  ME  charts 
and  even  the  mission  statement  itself  are  not  set  in  stone.  All  of 
these  things  should  be  considered  to  be  living  documents,  changing 
as  circumstances  within  and  without  the  organization  change  with 
their  features  being  especially  sensitive  to  the  spiraling 
expectations  of  customers.  This  is  especially  true  of  the  indi¬ 
cators  and  ME  charts.  As  processes  improve  (Weaver  &  Upton, 
1992a&b) ,  the  feasible  bests  and  worsts  will  change,  and  indicators 
may  even  become  "maxed  out"  such  that  they  are  no  longer  effected 
by  further  improvement  in  the  processes.  As  this  happens,  the 
target  organization  may  need  to  redo  some  or  all  of  the  TQM/MGEEM 
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structure  to  continually 
improve  its  value  as  a 
basis  for  improvement. 
These  decisions,  as  well 
as  the  use  of  the  ME 
charts  themselves 
through  the  philosophy 
of  the  Deming  Cycle 
(Fig.  17,  Deming  1986) 
is  the  final,  continuing 
phase  of  a  TQM/  MGEEM 
implementation  and  is 
discussed  in  detail  in 
Weaver  &  Upton  (1992b) . 


Using  the  ME  Charts  - 
the  Feedback  Team 


Figure  17  Deming  Cycle. 


On  a  periodic  basis,  usually  once  a  month  but  dictated  by  the 
measurement  period  of  the  indicators,  leadership  will  need  to  meet 
with  selected  customers,  suppliers  and  workers  to  review  the  ME 
chart  data.  It  is  at  this  time  that  review  and  changes  may  be  made 
in  the  TQM/MGEEM  system.  The  main  purpose  of  a  feedback  team, 
however,  is  to  review  the  information  posted  periodically  on  the  ME 
Charts  showing  how  the  organizations  processes  are  working  (See 
Weaver  &  Upton  (1992b)  and  "Phase  4:  Feedback"  in  Weaver,  Upton,  & 
Frank  (1992)  for  discussion  of  this  vital  part  of  TQM/MGEEM). 

The  first  several  feedback  meetings  may  take  place  without 
inviting  customers  or  suppliers  as  the  organizational  members 
conduct  sanity  checks  and  become  familiar  with  reviewing  the  ME 
Charts.  Later,  customers  will  need  to  be  invited  to  aid  in 
identification  of  customer  expectations.  Also  later,  invitations 
to  suppliers  will  be  necessary  to  aid  in  improving  the  quality  of 
the  organizations  inputs. 


Well  in  advance  of  a  projected  feedback  session,  the  represen¬ 
tative  customers  and  suppliers  should  be  contacted  and  a  time  set 
for  the  feedback  team  meeting.  This  can  be  in  conjunction  with  a 
regularly  scheduled  staff  meeting  or  can  be  completely  separate. 
In  addition  to  the  organization's  commander /manager  and  mid-level 
supervisors,  key  workers  should  also  attend  to  provide  a  forum  for 
their  unique  and  valuable  ideas  and  perspectives.  If  the  organiza¬ 
tion  is  small  enough  for  a  combination  of  the  Blue  and  Gold  teams, 
then  all  the  members  of  the  organization  might  be  able  to  partici¬ 
pate  in  these  sessions.  If  there  were  two  teams  and  the  organiza¬ 
tion  is  large,  workers  should  be  identified  on  a  rotating  basis  so 
that  all  interested  workers  will  have  a  chance  to  participate  at 
feedback  sessions.  In  this  context  the  relational  definition  of 
"workers"  used  for  the  Gold  Team  members  is  again  used  for  the 
feedback  team. 
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Feedback 


Feedback  comes  in  two  phases;  first  to  management,  and  second 
to  workers.  The  first  can  be  accomplished  in  a  normal  staff 
meeting,  the  second  will  be  most  effective  through  the  use  of  a 
feedback  team  meeting.  A  detailed  explanation  of  the  first  is 
beyond  the  scope  of  this  guide  and  can  be  found  in  "Feedback  in 
TQM/MGEEM"  in  Weaver  &  Upton  (1992b)  along  with  more  detail  about 
worker  feedback.  The  facilitator  for  these  sessions  may  or  may  not 
be  the  same  facilitator  as  was  used  for  the  development  process. 
Eventually,  these  sessions  may  be  facilitated  almost  entirely  by 
the  supervisor/commander  of  the  target  organization  itself.  For 
the  facilitator,  conducting  a  feedback  session  consists  mainly  of 
displaying  the  ME  charts  for  the  organization  with  the  results  for 
that  work  cycle  posted  on  them  and  leading  the  team  in  discussion 
as  to  the  areas  they  wish  to  improve  and  how  best  to  address  those 
concerns.  For  more  information  see  "How  feedback  teams  improve 
processes,"  Appendix  B,  Weaver  &  Upton  (1992b). 


Improving  the  System 

As  mentioned  at  the  beginning  of  this  section,  the  TQM/MGEEM 
measurement  system  should  not  be  set  in  stone.  From  time  to  time 
the  organization  will  find  it  necessary  to  alter  some  or  all  of  its 
components.  Any  part  of  the  system,  from  mission  statement  to  ME 
charts,  may  need  alterations  as  mission  needs,  customers,  suppli¬ 
ers,  technology,  circumstances,  and  TQM  knowledge  change.  Minor 
changes  may  be  accomplished  during  the  feedback  sessions,  with 
full-blown  Blue  or  Gold  Team  meetings  only  becoming  necessary  when 
major  revisions  are  needed.  It  is  paramount  to  remember  that,  like 
all  systems  within  the  organization,  the  measurement  system  must  be 
continually  improved  to  provide  higher  and  higher  quality  to 
management,  workers  and  customers. 
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^pendix  A;  A  Facilitator Checklist  for  the  TOM/MGEEM  Process. 

I .  Preparation 

A.  Familiarization  with  Target  Organization 

B.  Identify  members  of  Blue  and  Gold  team 

C.  Set  meeting  time/date 

D .  Secure  Room 

E.  Secure  Needed  Supplies 

II .  Blue  Team 

A.  Brief  overview  of  TQM/MGEEM  process  and  NGT 

B.  Review  Mission  Statement 

1.  Review  example  mission  statements 

2.  Alter  old  or  “straw  man"  mission  statement 

C.  Identify  Customers 

1.  Discuss  systems  diagram  (fig  1) 

2.  NGT  with  abbreviated  voting 

"Who  are  the  customers  of  _ " 

3 .  Pareto  chart  of  results 

4.  Discussion  of  "Critical"  vs  "Important"  customers. 

D.  Identify  Suppliers 

1.  Review  systems  diagram 

2.  NGT  with  abbreviated  voting 

"Who  are  the  suppliers  of  _ 

3.  Pareto  chart  of  results 

4.  Discussion  of  "Critical"  vs  "Important"  suppliers. 

E.  Identify  KRAs 

1.  Examples 

2.  NGT;  "What  categories  of  results  is  this 

organization  expected  to  accomplish?" 

"What  does  the  Air  Force  pay  us  to  do?" 

3.  Identify  KRAs  to  include  in  parsimonious  system. 

III.  Gold  Team 

A.  Brief  overview  of  TQM/MGEEM  process 

B.  Overlap  members  summarize  Blue  Team  work 

C.  Identify  Indicators  for  each  KRA 

1.  Brief  theory  of  measurement,  examples, 
characteristics  of  good  indicators 

2.  start  with  first  KRA 

3.  NGT:  "What  quantitative  indicator (s)  should  the 

manager  of  _  track  on  a  periodic  basis  to 

tell  whether  the  KRA  is  being  accomplished?" 

4.  Identify  indicator (s)  for  KRA 

5.  Define  specif ication/equat ion,  source,  feasible 
best,  feasible  worst,  and  indifference  point. 

6.  Choose  next  KRA,  repeat  steps  3-6  until  indicators 
are  identified  for  all  KRAs. 

D .  Rank  Indicators 

1.  Prepare  Indicator  Weighing  Table 

2.  Rank  and  assign  ME  Point  values  to  all  indicators 

E.  Develop  ME  Charts 
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^pendix  B:  Blank  ME  Chart 
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